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In a genetic algorithm, fluctuations of the entropy of a genome over time are interpreted as 
fluctuations of the information that the genome's organism is storing about its environment, being 
this reflected in more complex organisms. The computation of this entropy presents technical 
problems due to the small population sizes used in practice. In this work we propose and test 
an alternative way of measuring the entropy variation in a population by means of algorithmic 
information theory, where the entropy variation between two generational steps is the Kolmogorov 
complexity of the first step conditioned to the second one. As an example application of this 
technique, we report experimental differences in entropy evolution between systems in which sexual 
reproduction is present or absent. 
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I. INTRODUCTION 

The evolution over time of the entropy of a genome 
within a population is currently an interesting problem 
which is conjectured to be connected to the evolution 
of the complexity of organisms in a genetic algorithm 
[l|, 0. The complexity of the genome of an organism 
is considered to be the amount of information about its 
environment it stores. That is, evolution would cause 
the appearance of more complex sequences, which cor- 
respond to more complex phenotypes. This hypothesis 
states that natural selection acts as a Maxwell demon, 
accepting only those changes which adapt better to the 
environment and give rise to more complex individuals 
with genomes of lower entropy. This idea was tested by 
the simulation of a very simple system of asexual indi- 
viduals in fixed environmental conditions. 

However, it is well known that that the computation 
of the entropy as 

//(genome) = — p(x) logp(x) 

zEgcnomcs in the population 

(1) 

has technical complications, due to the large size of the 
sample needed to estimate it with accuracy [H [3, 0] ■ I n 
practice, it is usually estimated as 



//(genome) 



E 



H(i), 



(2) 



i— 1 . . sizc(gcnomc) 



i.e., as the sum of the entropy contributions of each lo- 
cus in the genome. This estimation misses the entropy 
contributions due to epistatic effects. Some sophisticated 
statistical methods can be used to remedy this (see Ap- 
pendix in jH), although we will not deal with them in 
this work. 

An still unexplored way to overcome this problem is 
to estimate the entropy of a genome as its average Kol- 
mogorov complexity 

{K (genome) ) gcnomos in thc population -»• //(genome) (3) 

(see (fj H, H[). However, this result only holds for in- 



finitely long sequences, and therefore it cannot be applied 
to finite (sometimes short) genomes. 

If we are only interested in the entropy evolution of 
the genome, and not in the particular value estimation, 
we can resort to the following trick: the genetic algo- 
rithm can be modelled as a thermodynamic system which 
evolves over time, where every population is just a mea- 
surement by an observer, i.e. the system is modeled as 
a statistical ensemble of genomes and each measurement 
is just a sample from that ensemble. 

Now we can measure the entropy evolution of the sys- 
tem from two different viewpoints. The first is the sys- 
tem itself, where the entropy is calculated a la Shannon 
(equation ^ by estimating the probabilities of the loci 
alleles, using the frequencies of the ensemble sample. 

The second way of measuring the entropy is from the 
viewpoint of the observer, where a measurement is made 
of the population at each time step, and the information 
about the system is updated, i.e., the observer measures 
the system at time t and stores this information St. At 
time t + 1 the observer makes another measurement and 
substitutes St by St+i- The entropy variation due to 
this substitution can be calculated for both equilibrium 
and non-equilibrium thermodynamic systems 0, IT3 , fl3 | . 
Since evolution cannot be modeled as a system in equi- 
librium, the second case applies: the mutation and gen- 
erational replacement operators may increase or decrease 
the entropy of the system [T3, EH ■ 

Thus the entropy variation from the observer view- 
point is K(St) — K(S t +i) bits. As K(-) is an incom- 
putable measure, we estimate it by using the Lempel-Ziv 
algorithm [T3| as \im n ^ oc (l/n)\LZ(x)\ = (l/n)K(x\n), 
where x is an infinite string and \LZ(x) \ is the size of the 
same string compressed [f| . Now our measurement of the 
system at time t, \St \ is much larger than with equation 
[3] (just the genome), so the estimation becomes possible. 



II. EXPERIMENTS 

We want to test whether the evolution of 
K (population sample) can help in the study of the 
evolution of //(genome). Both measurements have their 
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FIG. 1: Histogram of the 22 experimental correlation coeffi- 
cients. 

limitations, but their agreement would provide evidence 
that the entropy evolution is being studied correctly. 

We have evaluated this experimentally, using the ge- 
netic algorithm proposed by |6J], which is able to re- 
produce sexual behaviour in a vary detailed way, be- 
cause it includes several features absent in the [2j ex- 
periments, such as sexual reproduction, different inter- 
locus and intra-locus interactions across the genotypic or 
phenotipyc distance, and the evolutionary mechanisms of 
mutation and natural selection. 

We have implemented and run the same simulation 
proposed by Hayashi et al. The model's sexual dynam- 
ics can be summarized as follows: there are two different 
sexes (male and female). The likelihood that a female 
with trait a will mate with a male with trait b is defined 
by tpi*}) — e~~ ad , where d is the genetic or phenotypic dis- 
tance measuring compatibility between the sexes, and a 
is a parameter that represents the compatibility between 
traits. The value of a used in the simulations is 0.005. 
The overall number of offspring produced by a female 
in each coupling is given by Wf — B max e~ sc ( p ~ p ° pt * > , 
where P is the proportion of males with which the fe- 
male has been able to mate. The parameter P opt (which 
can take the values {0.2, 0.4, 0.6, 0.8}) defines the fertility 
of the female. The parameter sc (ranging from 1.02 to 
4 x 1.02) stands for sexual conflict selection in females. 
B m ax is the maximum possible number of offspring (5). 
The sex and the father are randomly chosen for each off- 
spring, and the number of males each female encounters 
is n = 20. 

Twenty two different experiments have been per- 
formed, if (population sample) and iJ(genome) have 
been estimated for each generation step with the methods 
described above. The system evolved for 10,000 genera- 
tions with a population size of 1,000 individual genomes. 

In all our results, both measures were highly correlated 
(see fig. [1]), giving evidence that the dual measurement 
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FIG. 2: Example of continuous evolutionary chase without 
genetic differentiation obtained with parameters B ma x = 5, 
Popt = 0.2, n = 5.0 x 10~ 5 , a = 0.05, sc = 1.02, 2 loci, pheno- 
typic model: additive. Experimental correlation coefficient: 
0.9956. 
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FIG. 3: Example of differentiation without speciation, ob- 
tained with parameters B ma x = 5, P op t = 0.8, fi = 5.0 x 10~ 5 , 
a = 0.05, sc — 1.02, 8 loci, phenotypic model: co-dominance. 
Experimental correlation coefficient: 0.9741. 

of the evolution of entropy by means of Kolmogorov com- 
plexity confirms the use of the Shannon entropy. 

III. DISCUSSION 

When sexual dynamics are introduced in the system, 
the increase of complexity observed by Q is not present 
anymore. 

The typical result observed in our experiments is a 
chaotic behavior of the entropy (fig. [2]). Only large 
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FIG. 4: Example of genetic differentiation without co- 
evolutionary chase or simpatryc speciation, obtained with pa- 
rameters Bmax = 5, P op t = 0.2, (j, = 5.0 X 10~ 5 , a = 0.05, 
sc = 1.02, 32 loci, phenotypic model: dominance. Experi- 
mental correlation coefficient: 0.8571. 



genome lengths or high female mating rates (P pt) es- 
cape from this (the other parameters seem to have little 
importance). The effect of this is to increase the auto- 
correlation of the entropy time series and provoke the 
rise of a few entropy bumps during a small number of 
generations (figs. [3] and [4]). 

Our hypothesis for this behavior of the entropy is the 
absence of natural selection in the Hayashi et al. @ 
model, which could explain the similarities between male 
and female evolution (fig. [5J. Without natural selection, 
the environment for females is reduced to random bound- 
ary conditions (mutations). On the other hand, males are 
selected by females as mating partners. In this way, fe- 
males can be considered to become the environment for 
males, since they determine the way in which the entropy 
of the males evolves. On the other hand, females have no 
environment to adapt to. Perhaps if the pressure of nat- 
ural selection was applied both to male and female (not 
necessarily in the same way), more complex patterns in 
the behavior of their entropies would appear. The fact 
that natural selection is not taken into account may be 
the cause of the differences in entropy evolution between 
the models by @ and @ , and the reason why global de- 
creases in entropy are not observed in the latter. 



tropy evolution via Kolmogorov Complexity. This over- 
comes many limitations that arise from epistatic effects 
between loci and provide an easy way to do study the dy- 
namics without resorting to complex mathematical trick- 
H(genome) female 
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FIG. 5: Same parameters as in fig. 2 with entropy calculation 
decomposed by sex. 



eries. We also use this methodology to study what is the 
effect of sexual reproduction in terms of the evolution of 
complexity, as a decrease of entropy. We show that when 
sexual reproduction is present the population enters in 
a chaotic regime of complexity driven by the complex- 
ity drifts of the female organisms. We suggest that this 
might be cause for female organism evolving chaotically 
without natural selection, and male organisms evolving 
with females as the boundary conditions, which gives an 
overall chaotic evolution of complexity. The next imme- 
diate step is to introduce natural selection in the exper- 
iments and study whether this will change the evolution 
of complexity. We plan to do it by implementing the typ- 
ical natural selection operators from genetic algorithms 
such as tournament selection, steady state-selection and 
so forth. We conjecture that introducing natural selec- 
tion will remove the chaotic complexity dynamics and 
might probably get closer to what Hayasi et. al. formerly 
reported: an increase in complexity by removing genetic 
mutations that do not improve the fitness function. 



IV. CONCLUSSIONS AND FUTURE WORK 
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